Enable use of nullable maps for headers, params and query #694

kamperiadis · 2023-01-20T21:25:42Z

BREAKING CHANGE: Enabled the UseNullableValueDictionaryForHttp capability which makes the host send back request information in nullable maps (i.e. When the UseNullableValueDictionaryForHttp capability is enabled, the host stores the values in nullable_headers, nullable_params and nullable_query instead of headers, params and query)

Issue describing the changes in this PR

Context:
This is the sample function app code run for the screenshots below:

Before:
Without this capability, if the customer were to set an empty string as the query value, the host would not add the entry to the query map. This means that if the customer were to try to access that entry from the query map (via the getQueryParameters function), they would just get null.

After:
With this capability, if the customer were to set an empty string as the query value, the host would add the entry to the query map with the value just being the empty string.

This is meant to resolve issue #683

Pull request checklist

My changes do not require documentation changes
- Otherwise: Documentation issue linked to PR
My changes should not be added to the release notes for the next release
- Otherwise: I've added my notes to release_notes.md
My changes do not need to be backported to a previous version
- Otherwise: Backport tracked by issue/PR #issue_or_pr
I have added all required tests (Unit tests, E2E tests)

kaibocai · 2023-01-23T22:19:44Z

src/main/java/com/microsoft/azure/functions/worker/binding/RpcHttpRequestDataSource.java

+		this.fields = Arrays.asList(
+				convertFromNullableMap(this.httpPayload.getNullableHeadersMap()),
+				convertFromNullableMap(this.httpPayload.getNullableQueryMap()),
+				convertFromNullableMap(this.httpPayload.getNullableParamsMap()));


not related to this PR, but do we know difference between query map and param map?

Looking at the host code, it looks like the three key differences between query map and param map are that:

They come from different parts of the request object (i.e. request.Query vs request.HttpContext.Items.TryGetValue(HttpExtensionConstants.AzureWebJobsHttpRouteDataKey, out object routeData))

We do not add null values to the parameters map, even if the UseNullableValueDictionaryForHttp capability is on

We add empty strings to the parameters map, even if the UseNullableValueDictionaryForHttp capability is off

kaibocai · 2023-01-23T22:25:24Z

...st/java/com/microsoft/azure/functions/worker/binding/tests/RpcHttpRequestDataSourceTest.java

@@ -41,14 +56,150 @@ public static RpcHttp getTestRpcHttp(Object inputBody) throws Exception {
    return httpBuilder.build();
  }

+  @Test
+  public void rpcHttpDataSource_To_HttpRequestMessage_NullableQueryParamsEmpty_EnvSettingEnabled() throws Exception {


let's use camel-case syntax for method name, the old tests were wrote by c# expert, seems they are using c# name convention.

kaibocai · 2023-01-23T22:37:33Z

...st/java/com/microsoft/azure/functions/worker/binding/tests/RpcHttpRequestDataSourceTest.java

+            parameters[0].getParameterizedType());
+    BindingData actualArg = actualBindingData.orElseThrow(WrongMethodTypeException::new);
+    HttpRequestMessage<?> requestMsg = (HttpRequestMessage<?>) actualArg.getValue();
+    assertEquals(requestMsg.getQueryParameters().get("name"), "");


let's follow the convention that expect value comes first and test value comes second. I think we can use assertEmpty here.

kaibocai · 2023-01-23T22:38:06Z

...st/java/com/microsoft/azure/functions/worker/binding/tests/RpcHttpRequestDataSourceTest.java

+            parameters[0].getParameterizedType());
+    BindingData actualArg = actualBindingData.orElseThrow(WrongMethodTypeException::new);
+    HttpRequestMessage<?> requestMsg = (HttpRequestMessage<?>) actualArg.getValue();
+    assertEquals(requestMsg.getQueryParameters().get("name"), null);


we can use assertNull

kaibocai · 2023-01-23T22:53:59Z

...st/java/com/microsoft/azure/functions/worker/binding/tests/RpcHttpRequestDataSourceTest.java

+  }
+
+  @Test
+  public void rpcHttpDataSource_To_HttpRequestMessage_NullableQueryParamsNonEmpty() throws Exception {


I think we can have the test cases more clear. There are two major scenarios we want to test

appsetting to false

appsettting to true

each of those scenarios we want to test

normal case -- param value empty

param value empty

So we can have two test cases in total

with appsetting to true

have one normal param -- expect same value

have one param with empty value -- expect empty string

with appsetting to false

have one normal param -- expect normal value

have one param with empty value -- expect null

We can test header map and query map in same test method.
So we can reduce the test methods number to 2 but still have full coverage. Let me know if this make sense or not. Thanks.

kaibocai · 2023-01-26T14:32:57Z

Once this feature is released, need to document it somewhere (repo wiki?) for customer. Maybe create an issue to track creating the document.

shreyas-gopalakrishna

LGTM! Just one minor refactor

shreyas-gopalakrishna · 2023-01-26T16:43:45Z

src/main/java/com/microsoft/azure/functions/worker/binding/RpcHttpRequestDataSource.java

@@ -86,4 +91,16 @@ public Builder createResponseBuilder(HttpStatus status) {
 			return new HttpRequestMessageImpl(v, bodyData.getValue());
 		});
 	}
+
+	private static Map<String, String> convertFromNullableMap(Map<String, NullableTypes.NullableString> nullableMap) {
+		if (Util.isTrue(System.getenv("FUNCTIONS_WORKER_NULLABLE_VALUES_ENABLED"))) {


Let us make FUNCTIONS_WORKER_NULLABLE_VALUES_ENABLED a constant.

nice catch! there is a constant file you can define it there.

Enable use of nullable maps for headers, params and query

ea4df5f

kamperiadis requested review from anirudhgarg, TsuyoshiUshio, amamounelsayed, kaibocai and shreyas-gopalakrishna as code owners January 20, 2023 21:25

Check app setting and filter map

5c74367

kaibocai reviewed Jan 23, 2023

View reviewed changes

kaibocai linked an issue Jan 24, 2023 that may be closed by this pull request

Empty string cannot be passed in HttpTrigger query string #683

Open

Simplify unit tests

1e53683

kaibocai previously approved these changes Jan 26, 2023

View reviewed changes

shreyas-gopalakrishna previously approved these changes Jan 26, 2023

View reviewed changes

Add app setting name in constants file

83d5388

kamperiadis dismissed stale reviews from shreyas-gopalakrishna and kaibocai via 83d5388 January 27, 2023 16:02

kamperiadis added 2 commits January 27, 2023 10:02

Merge branch 'dev' into kamperiadis/nullableMap

60bb55a

Merge branch 'dev' into kamperiadis/nullableMap

727323c

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Enable use of nullable maps for headers, params and query #694

Enable use of nullable maps for headers, params and query #694

kamperiadis commented Jan 20, 2023

kaibocai Jan 23, 2023

kamperiadis Jan 25, 2023

kaibocai Jan 23, 2023

kaibocai Jan 23, 2023

kaibocai Jan 23, 2023

kaibocai Jan 23, 2023

kaibocai commented Jan 26, 2023

shreyas-gopalakrishna left a comment

shreyas-gopalakrishna Jan 26, 2023

kaibocai Jan 26, 2023

Enable use of nullable maps for headers, params and query #694

Are you sure you want to change the base?

Enable use of nullable maps for headers, params and query #694

Conversation

kamperiadis commented Jan 20, 2023

Issue describing the changes in this PR

Pull request checklist

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kaibocai commented Jan 26, 2023

shreyas-gopalakrishna left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment